What is the structure of your dataset?

There are 1599 observations of red wines, and 13 total variables. Of these, only 12 are relevant, as the variable “X” is an identifier. The variable “quality” is the dependent variable of interest, since we are interested in predicting “quality” using the other 11 variables which measure chemical properties of the red wines.

• All 11 chemical property variables are numeric, there are no categoric variables.
  
• There are no missing / NA values in the dataset.
• The distributions are either normal (“quality”, “volatile.acidity”, “chlorides”, “density”, “pH”) or right-skewed (“fixed.acidity”, “citric.acid”, “residual.sugar”, “free.sulfur.dioxide”, “total.sulfur.dioxide”, “sulphates”, “alcohol”)

What is/are the main feature(s) of interest in your dataset?

The main feature of interest in the red wines dataset is the “quality” score. We are interested in seeing which of the other 11 features are related to the “quality”, and also in creating a predictive model for “quality” based on the relevant variables out of the 11.

What other features in the dataset do you think will help support your investigation into your feature(s) of interest?

Not knowing much about wines or alcohol, each of the 11 chemical variable properties is equally likely to influence the “quality” score. We will find out more in the bivariate investigation section.

Did you create any new variables from existing variables in the dataset?

The new categoric variable “quality.factor” was created from the integer/numeric variable “quality”. This is justified since “quality” only has discrete integer values.

The new categoric variable “quality.rating” was also created from “quality”, but this time divides the score into ratings of “low” (0 - 4), “average” (5 - 6), and “high” (7 - 10).

Of the features you investigated, were there any unusual distributions? Did you perform any operations on the data to tidy, adjust, or change the form of the data? If so, why did you do this?

This was a relatively tidy dataset with no missing values, and no cleaning operations were necessary. The only adjustments were:

• adding the “quality.factor” and “quality.rating” categoric variables to use in later boxplots
  
• showing additional plots limited to the 95% percentile values for “residual.sugar” and “chlorides”, and 99% percentile for “free.sulfur.dioxide”, “total.sulfur.dioxide”, and “sulphates” to get a better look at the distribution.

Talk about some of the relationships you observed in this part of the investigation. How did the feature(s) of interest vary with other features in the dataset?

From the scatterplots with boxplot overlays and the correlation analysis, we can make some generalisations about what factors are more likely to affect the “quality” measure:

• factors that appear to affect “quality” (strong trend) :

  • volatile.acidity (-0.391)
    
  • citric.acid (0.226)
    
  • sulphates (0.251)
    
  • alcohol (0.476, highest correlation factor)

• factors that might affect “quality” (slight trend) :

  • fixed.acidity (0.124)
    
  • chlorides (-0.129)
    
  • total.sulfur.dioxide (-0.185)
    
  • density (-0.175)
    
  • pH (even though the correlation coefficient is low at -0.058)

• factors that do not appear to affect “quality” (no trend or nonlinear trend) :

  • residual.sugar (0.014)
    
  • free.sulfur.dioxide (-0.051)

Often, it was easier to see a relationship on the “quality.rating” plots which only have 3 levels compared to the “quality” plots with 6 levels

Did you observe any interesting relationships between the other features (not the main feature(s) of interest)?

There are some chemical property variables which are strongly correlated with each other, which makes sense since they measure properties which are closely related:

• “free.sulfur.dioxide” and “total.sulfur.dioxide”
• “fixed.acidity”, “volatile.acidity”, “citric.acid” and “pH”

Some strong relationships probably have a chemical explanation:

• “fixed.acidity”, “volatile.acidity”, “citric.acid”, “pH” and “density”
• “citric.acid” and “sulphates”
• “chlorides” and “sulphates”

What was the strongest relationship you found?

These were the strongest correlations among all the variables:

• pH : citric.acid (-0.542) - not surprising for the neative correlation since pH is related to acidity (less pH = more acidity)
  
• pH : fixed.acidity (-0.683) - same as above
  
• density : fixed.acidity (0.668) - there must be a chemical explanation for this, which is that acid is more dense than water
  
• total.sulfur.dioxide : free.sulfur.dioxide (0.668) - not surprising for the strong positive correlation, both are sulfur measures
  
• citric.acid : volatile.acidity (-0.552) - not surprising that the acid level is related to an acidity measure, but surprising that the coefficient is negative
  
• citric.acid : fixed.acidity (0.672) - the coefficient sign is positive, surprising since it is negative for “volatile.acidity”

The strongest correlations compared to “quality” are:

• quality : alcohol (0.476)
  
• quality : sulphates (0.251)
  
• quality : citric.acid (0.226)
  
• quality : volatile.acidity (-0.391)

Linear Regression Model

Let’s create a linear regression model to predict the “quality” score using all the chemical property variables (full model):

## 
## Call:
## lm(formula = quality ~ ., data = red_subset2)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -2.68911 -0.36652 -0.04699  0.45202  2.02498 
## 
## Coefficients:
##                        Estimate Std. Error t value Pr(>|t|)    
## (Intercept)           2.197e+01  2.119e+01   1.036   0.3002    
## fixed.acidity         2.499e-02  2.595e-02   0.963   0.3357    
## volatile.acidity     -1.084e+00  1.211e-01  -8.948  < 2e-16 ***
## citric.acid          -1.826e-01  1.472e-01  -1.240   0.2150    
## residual.sugar        1.633e-02  1.500e-02   1.089   0.2765    
## chlorides            -1.874e+00  4.193e-01  -4.470 8.37e-06 ***
## free.sulfur.dioxide   4.361e-03  2.171e-03   2.009   0.0447 *  
## total.sulfur.dioxide -3.265e-03  7.287e-04  -4.480 8.00e-06 ***
## density              -1.788e+01  2.163e+01  -0.827   0.4086    
## pH                   -4.137e-01  1.916e-01  -2.159   0.0310 *  
## sulphates             9.163e-01  1.143e-01   8.014 2.13e-15 ***
## alcohol               2.762e-01  2.648e-02  10.429  < 2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.648 on 1587 degrees of freedom
## Multiple R-squared:  0.3606, Adjusted R-squared:  0.3561 
## F-statistic: 81.35 on 11 and 1587 DF,  p-value: < 2.2e-16

##        fixed.acidity     volatile.acidity          citric.acid 
##             7.767512             1.789390             3.128022 
##       residual.sugar            chlorides  free.sulfur.dioxide 
##             1.702588             1.481932             1.963019 
## total.sulfur.dioxide              density                   pH 
##             2.186813             6.343760             3.329732 
##            sulphates              alcohol 
##             1.429434             3.031160

In the summary, we can see which variables are significant to the model:

• volatile.acidity ***
  
• chlorides ***
  
• free.sulfur.dioxide *
  
• total.sulfur.dioxide ***
  
• pH *
  
• sulphates ***
  
• alcohol ***

Though there are many significant variables, the adjusted R-squared is relatively low at 0.3561, which means that it is not a great predictor for variations in the output variable.

Several variables have high VIF values, which means that there is multicollinearity among the variables of the full model.

For the next model iteration, we will decrease the number of predictor variables by only including the variables that were marked signifcant from the full model’s summary.

## 
## Call:
## lm(formula = quality ~ volatile.acidity + chlorides + total.sulfur.dioxide + 
##     free.sulfur.dioxide + pH + sulphates + alcohol, data = red_subset2)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -2.68918 -0.36757 -0.04653  0.46081  2.02954 
## 
## Coefficients:
##                        Estimate Std. Error t value Pr(>|t|)    
## (Intercept)           4.4300987  0.4029168  10.995  < 2e-16 ***
## volatile.acidity     -1.0127527  0.1008429 -10.043  < 2e-16 ***
## chlorides            -2.0178138  0.3975417  -5.076 4.31e-07 ***
## total.sulfur.dioxide -0.0034822  0.0006868  -5.070 4.43e-07 ***
## free.sulfur.dioxide   0.0050774  0.0021255   2.389    0.017 *  
## pH                   -0.4826614  0.1175581  -4.106 4.23e-05 ***
## sulphates             0.8826651  0.1099084   8.031 1.86e-15 ***
## alcohol               0.2893028  0.0167958  17.225  < 2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.6477 on 1591 degrees of freedom
## Multiple R-squared:  0.3595, Adjusted R-squared:  0.3567 
## F-statistic: 127.6 on 7 and 1591 DF,  p-value: < 2.2e-16

##     volatile.acidity            chlorides total.sulfur.dioxide 
##             1.241819             1.333333             1.943920 
##  free.sulfur.dioxide                   pH            sulphates 
##             1.882706             1.254570             1.321931 
##              alcohol 
##             1.220157

This second model has significance of close to 0 for all the included predictor variables, except for “free.sulfur.dioxide” with significance of * (<= 0.01). Adjusted R-squared has increased slightly, to 0.3567

The VIF values are now lower (< 2) for all the variables in the model.

Let’s try removing “free.sulfur.dioxide” in the next model since we already included “total.sulfur.dioxide” (which is highly correlated with “free.sulfur.dioxide”).

## 
## Call:
## lm(formula = quality ~ volatile.acidity + chlorides + total.sulfur.dioxide + 
##     pH + sulphates + alcohol, data = red_subset2)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -2.60575 -0.35883 -0.04806  0.46079  1.95643 
## 
## Coefficients:
##                        Estimate Std. Error t value Pr(>|t|)    
## (Intercept)           4.2957316  0.3995603  10.751  < 2e-16 ***
## volatile.acidity     -1.0381945  0.1004270 -10.338  < 2e-16 ***
## chlorides            -2.0022839  0.3980757  -5.030 5.46e-07 ***
## total.sulfur.dioxide -0.0023721  0.0005064  -4.684 3.05e-06 ***
## pH                   -0.4351830  0.1160368  -3.750 0.000183 ***
## sulphates             0.8886802  0.1100419   8.076 1.31e-15 ***
## alcohol               0.2906738  0.0168108  17.291  < 2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.6487 on 1592 degrees of freedom
## Multiple R-squared:  0.3572, Adjusted R-squared:  0.3548 
## F-statistic: 147.4 on 6 and 1592 DF,  p-value: < 2.2e-16

##     volatile.acidity            chlorides total.sulfur.dioxide 
##             1.227967             1.332977             1.053830 
##                   pH            sulphates              alcohol 
##             1.218707             1.321237             1.218733

In this third model, all 6 predictors have significance of close to 0. However, the adjusted R-squared has decreased slightly to 0.3548. So we know that the 6 predictors are very likely to influence the quality score, but the explantory level of our model is still not good. There are likely to be variables we are missing that are needed in order to create a better model with more explanatory power.

The VIF values are lower still, now all < 1.5 which means a low level of multicollinearity.

## [1] "summary : actual quality values"

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##   3.000   5.000   6.000   5.636   6.000   8.000

## [1] "summary : predicted quality values from model 2"

## Length  Class   Mode 
##      0   NULL   NULL

## [1] "quality rating distribution"

##     low average    high 
##      63    1319     217

This is a plot of the error (predicted - actual) values from using model 2 which had the highest adjusted R-squared. We can see that from the plot and the summary tables that the model overpredicts for low scores and underpredicts for high scores (min 4.304 in prediction versus min 3 in actual, max 7.342 in prediction versus max 8 in actual). This clustering around the middle/average region of 5-6 scores likely is due to the concentration of our data points in this range - 1319 out of 1499 (87.99%) of our datapoints are in the middle/average range.

Now let’s look at the density plots of each of these variables from model_2 to see if the trends are noticeable and match with the coefficients from the linear model:

We multiplot some density plots for the variables in the linear model. Even though these are bivariate plots, the variables were decided as a result of running the linear regression models.

We can see the direction and strength of trends - for example, the positive / negative effects of “volatile.acidity” and “sulphates” are more obvious than for “total.sulfur.dioxide” and “free.sulfur.dioxide” whose plots don’t really show any trend.

Talk about some of the relationships you observed in this part of the investigation. Were there features that strengthened each other in terms of looking at your feature(s) of interest?

From the bivariate analysis, we saw that “alcohol” has the highest correlation with the “quality” score. So we explored the other variables compared to “alcohol”, and splitting or faceting by “quality.rating”. We saw some noticeable trends associated with the higher ratings:

• higher sulphates
  
• higher citric.acid
  
• lower volatile.acidity
  
• higher fixed.acidity
  
• lower chlorides
  
• higher density
  
• lower pH

Very small effects from:

• total.sulfur.dioxide

Were there any interesting or surprising interactions between features?

• “fixed.acidity” has low correlation with “quality” (0.124), and almost no correlation with “alcohol” (-0.062), but when we plot it against “alcohol” we can see a positive trend - with constant alcohol value, higher fixed acidity values usually have higher quality ratings. Probably due to the negative relationship that “alcohol”" has with “volatile acidity” (-0.202) and the negative correlation between “fixed.acidity” and “volatile.acidity” (-0.256)
  
• “density” has negative correlation with “quality” (-0.175), but when plotted against alcohol, higher density values have higer quality ratings. Likely due to the strong negative correlation between “alcohol” and “density” (-0.496)
  
• “total.sulfur.dioxide” and “free.sulfur.dioxide” have low correlations with “quality”" (-0.185, -0.051), but still appear in the linear model

OPTIONAL: Did you create any models with your dataset? Discuss the strengths and limitations of your model.

There were 3 linear models created - a full model (model_1), a model removing the non-significant factors from the full model (model_2), and a model removing an additional factor “free.sulfur.dioxide” (model_3).

The model with the highest adjusted R-squared (meaning explantory power) was model_2, though this value was low at 0.3567. But VIF analysis showed that the 7 factors in this model did not have a high degree of multicollinearity, so we use these variables which are all significant:

• volatile.acidity
  
• chlorides
  
• total.sulfur.dioxide
  
• free.sulfur.dioxide
  
• pH
  
• sulphates
  
• alcohol

Model strength(s): The low significance values show that these chemical properties are very likely related to the quality score, which is a goal of the exploration (to see which variables influence the quality)

Model weakness(es): As a linear regression model, the output is a numeric number with decimals even though the quality scale is a discrete integer. The explanatory power is low (low adjusted R-squared). We have a lot of data (~88%) with average quality scores (5, 6), but not a lot of data at the low and high ends (<= 4, >= 7) which means our model is not very good predicting low and high values - as seen in the error plot where it overestimates bad quality wines and underestimates good quality wines.

Plot One

Description One

This is a summary of the 11 chemical properties across the 3 subjective quality ratings (“quality.rating” - “low”, “average”, “high”) in a combined scatterplot and boxplot. The extreme outliers were removed by limiting the y-axis to the 95% percentile values for “residual.sugar” and “chlorides”, and to the 99% percentile values for “free.sulfur.dioxide”, “total.sulfur.dioxide”, and “sulphates”. We knew about the outliers for these 5 variables from the univariate exploration phase.

As we move across each individual plot from left to right, from quality rating “low” to “high”, we can see that some linear trends are more obvious, such as the negative trends for “volatile.acidity” and “pH”, and the positive trends for “citric.acid” and “alcohol”, while there are non-linear trends for “free.sulfur.dioxide” and “total.sulfur.dioxide”, and almost no trend for “residual.sugar”.

Because of the scatterplot, we can also see that most of our values are in the average “quality.rating” group, and that we have sparser data for low and high ratings.

Plot Two

Description Two

This is a density plot of the variables that were deemed significant by the linear model for “quality” (model 2 in the multivariate analysis section), coloured by the 3 “quality.rating” categories.

We can observe the distribution of these variables among the quality ratings and form some conclusions:

If a trend direction was seen in these density plots, the direction was also reflected in the model coefficient of the linear model. For example, the density plot of “volatile.acidity” shows a strong negative trend where higher quality wines have lower volatile acidity values. This negative trend matches the negative model coefficient.

All the variable trends in the plots here match with the direction of the linear model coefficients, which is expected.

Plot Three

Description Three

we plot the 6 chemical property variables from linear regression model 2 (excluding “alcohol”) against “alcohol” and “quality.rating” to see the influence of their values on the rating score, holding alcohol constant.

The 6 variables are:

• chlorides
  
• volatile.acidity
  
• sulphates
  
• pH
• free.sulfur.dioxide
  
• total.sulfur.dioxide

We also check the directions of the trends compared to “alcohol” (correlation values compared to alcohol, and the trend direction from the plot with alcohol and quality rating) and “quality” (correlation values compared to quality, and the coefficients from the linear model) :

Points of note:

• Generally, the signs on the correlation coefficients of the variables compared to “alcohol” and “quality”, along with the direction of the trend when plotted with “alcohol” and “quality” matches, except for 2 variables - “pH” and “free.sulfur.dioxide”
  
• “pH” has positive correlation with “alcohol”, but when plotted against both “alcohol” and “quality.rating”, we see that for constant alcohol value, lower pH values are associated with higher quality ratings. This can be explained by the negative effect of “pH” on “quality” (even though the correlation to “quality” is small at -0.058, there is a rather large -0.483 coefficient calculated from the linear model).
  
• “free.sulfur.dioxide” has a weak negative correlation with “alcohol” (-0.069) and a weak negative correlation with “quality” (-0.051), but in the linear model ends up with a small positive coefficient (0.0051)

Where did I run into difficulties in the analysis?

The main difficulty was trying to build a good predictive model with the existing data, which ultimately failed since the explanatory power of the final model was low (model_2 had the highest adjusted R-squared at 0.3567). Also, the linear regression model predicts a numeric number with decimals, while the “quality” score is a discrete integer.

Also, since there was not a lot of data outside of the “average” quality score range (~88% of observations were “average” quality), it was hard to infer trends / make predictions for lower and higher quality cases.

Where did I find successes?

We were successful at least in identifying variables which are correlatd with the output variable “quality” (from the correlation plots) and which likely influence the “quality” score (have significance close to 0 in the linear regression model). The scatterplots and geom_smooth lines were very helpful with showing patterns in the data.

However, all this is with the caveat that correlation does not equal causation, and that we are working with a dataset limited in observations (unequal distribution among the quality ratings) and characteristics (other objective or subjective properties that were not included).

What was surprising?

• That the adjusted R-squared ended up being so low (0.3567), even though the variables were significant
  
• Even though “total.sulfur.dioxide” and “free.sulfur.dioxide” have low correlations with “quality”" (-0.185, -0.051), they are still included in the final model

How could the analysis be enriched in future work?

The analysis can be improved by getting additional information about the existing wines, and getting additional observations of lower (< 5) and higher (> 6) wines, even though this was not part of the provided dataset.